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SIGNIFICANCE  AND  EXPLANATION 


A  common  problem  in  the  analysis  of  stochastic  systems  is  the  estimation 
of  a  stochastic  process  given  only  noise-corrupted  or  incomplete  observations. 
Examples  occur  in  communications  theory  when  one  wants  to  estimate  a  signal 
sent  over  a  noisy  channel  or  in  time  series  problems.  If  x(t)  is  a 
stochastic  process  denoting  the  signal,  the  observations  are  typically 
modelled  by 

y(t)  =  /J  h(x(8))da  +  dW(t)  , 

where  W(t)  is  an  independent  increments  "noise"  process,  usually  Brownian 
motion.  The  problem  of  filtering  is  to  build  an  estimate,  l.e.,  filter,  of 
x(t)  using  the  observations  y(s),  s  <  t.  Theoretical  characterizations  of 
best  mean-square  estimates  are  known,  but  can  be  translated  into  effective 
solutions  only  in  special  instances.  In  this  paper,  the  general  filtering 
problem  is  treated  by  attempting  to  expand  filters  in  series  of  multiple 
stochastic  integrals  of  the  form 

Two  primary  issues  raised  by  this  idea  are  considered;  representation  of  the 
optimal  mean-square  estimate  by  multiple  integral  expansions,  and  construction 
of  suboptimal  estimates  using  a  finite  number  of  multiple  integrals.  It  is 
shown  that  expansion  of  the  optimal  filter  is  indeed  possible,  and  a  method  is 
presented  for  finding  best,  finite  expansion  estimates.  A  rudimentary  algebra 
of  multiple  integral  expansions  is  first  developed  as  a  tool  to  prove  these 
results. 

The  responsibility  for  the  wording  and  views  expressed  in  this  descriptive 
summary  lies  with  MRC,  and  not  with  the  author  of  this  report. 


MULTIPLE  INTEGRAL  EXPANSIONS  FOR  NONLINEAR  FILTERING* 

Daniel  Ocone 

U 1  Introduction 

In  the  additive  noise  model  of  filtering.  Information  about  a  stochastic  process 
x(t),  t  >  0,  called  the  signal.  Is  received  through  observations  of  the  form 

y(t)  =•  h(x(a))ds  +  w(t)  t  >  0 

w(t)  Is  a  noise  term  that  corrupts  the  signal,  and  It  Is  usually  assumed  to  be  a  Brotmian 
motion.  The  filtering  problem  is  to  estimate  from  the  observations  y<s),  0  <  s  <  t,  a 
given  moment  f(x(t))  of  the  signal  at  time  t,  and.  If  estimators  minimizing  mean- 
square-error  are  desired,  this  means  calculating  the  conditional  mean  F{f(x(t))  IF 
F^  s  =■  cCyls)  I  0  <  s  <  t}.  E{f(x(t))  1  F^}  is  henceforth  referred  to  as  the  optimal 

filter.  Two  fundamental  characterizations  of  the  optimal  filter  are  available:  a)  a  Bayes 
formula  for  E{f(x(t))  I  F^)  as  the  ratio  of  two  conditional,  functional  Integrals 
<Kalllanpur,  Striebel  (B] ,  cf.  Hi. 2  of  this  paper):  b).  In  the  case  that  x(t)  Is  Martov, 
a  representation  of  the  optimal  filter  as  a  stochastic  Integral  against  the  Innovations 
process,  v(t)  y(t)  -  E(h(x(8))  I  f^}ds,  the  stochastic  Integrand  being  adapted  to 

the  observation  process  (Fujlsakl,  Kalllanpur,  and  Kunlta  [2]).  However,  though 
theoretically  deep,  these  results  lead  to  explicit  and  analytically  computable  solutions 
only  In  special  Instances. 


• 

This  work  formed  part  of  the  author's  Ph.D.  thesis  In  Applied  Mathematics  at  M.I.T.  under 
the  supervision  of  Professor  Sanjoy  K.  Mltter. 
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This  paper  studies  the  application  of  multiple  stochastic  Integral  expansions  to  the 


filtering  problem.  Any  filter,  optimal  or  suboptimal  Is  actually  an  anticipating 
functional  of  the  observation  process,  thus  suggesting  that  filters  be  represented  and 
analyzed  within  a  freunework  for  functional  expansions.  Multiple  stochastic  integrals  prove 
useful  for  this  purpose.  In  fact,  their  definition  originates  In  Wiener's  homogeneous 
chaos  theory,  which  constructs  orthogonal  decompositions  of  spaces  of  finite-variance 
functionals  of  Gaussian  processes  (cf.  Kalllanpur  [8]  and  Hlda  [5]).  In  the  Brownian 
motion  case,  each  subspace  of  the  decomposition  corresponds  to  the  apace  of  multiple 
stochastic  Integrals  of  a  given  order,  and,  thus,  Wiener's  theory  shows  that  any  L  - 
functional  of  the  Brownian  motion  may  be  expanded  in  a  series  of  multiple  integrals. 
Multiple  Integrals  have  been  used  already  to  solve  a  number  of  specific  estimation 
problems.  Marcus,  Mltter,  and  Ocone  (13]  apply  the  homogeneous  chaos  theory  to  compute 
conditional  statistics  of  polynomial  functionals  of  a  Gauss-Markov  process  observed  In 
white  noise,  and  Hlda  and  Kalllanour  [6]  use  multiple  Integrals  to  predict  non-linear 
functions  of  Brownian  signals  given  perfect  observations.  In  cumulant  approximations  of 
the  conditional  density  In  filtering,  Eterno  [1]  also  derives  expressions  using  multiple 
Integrals.  Here,  we  seek  to  apply  multiple  Integrals  of  the  form 


a(t,s^ 


,8  )dy(8  >«»»dy(s  )  , 

n  n  1 


where  a(...)  Is  deterministic,  to  the  general  filtering  problem.  We  focus  on  two  basic 
Issues;  the  expansion  of  the  optimal  filter  by  expressions  Involving  multiple  Integrals, 
and  the  construction  of  best  suboptimal  filters  having  a  finite  multiple  Integral  expansion 
of  specified  order.  It  Is  Important  to  observe  that  the  stochastic  Integrals  we  employ  are 
formed  from  the  observation  process  and  not  the  Innovations  process.  At  first.  Integration 
against  Innovations  might  appear  to  be  an  attractive  Idea  because  the  innovations  process 
is  Brownian,  Integrals  of  different  orders  are  thus  orthogonal,  and  homogeneous  chaos 
theory  can  be  applied.  However,  In  practice  the  Innovations  process  Is  not  available  since 
Its  construction  requires  the  estimate  E{h(x(t))  I  F^},  to  compute  which  Is  generally  a 
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difficult  filtering  problem  Itself.  Integrals  using  y(*)  directly  are  thus  more  natural, 
but,  due  to  their  more  general,  usually  non-Gaussian  character  are  more  difficult  to 
apply,  for  example,  in  auboptimal  estimation  one  might  like  to  project  random  variables  on 
a  sum  of  spaces  of  multiple  Integrals.  This  is  easily  done  for  Brownian  integrals,  using 
the  orthogonality  of  different  order  Integrals  and  explicit  formulae  to  calculate  the 
integrands,  but  not  so  easily  for  more  general  Integrals,  where  the  orthogonality  structure 
and  kernel  formulae  are  lost.  In  this  paper  we  describe  a  method  for  analysing  y( •)  - 
based  integrals,  that,  in  particular,  allows  resolution  of  this  projection  problem. 

The  paper  is  organized  as  follows.  $1.2  introduces  the  precise  filtering  model  we 
consider  and  recalls  the  Kallianpur-Strlebel  formula  for  the  optimal  estimate.  \  central 
feature  of  this  formula  is  the  fact  that  the  y( •)  process  is  absolutely  continuous  with 
respect  to  Brownian  motion.  Transformations  of  measure  so  that  y( •)  becomes  Brownian 
will  be  an  underlying  component  of  our  analysis  of  y(«)-based  Integrals.  $2  is  a  self- 
contained  treatment  of  multiple  integrals  of  Brownian  and  observation  processes.  We  define 
multiple  stochastic  integrals,  prove  technical  lemmas  for  later  use,  and  develop  some 
useful  properties  of  the  integrals.  Of  particular  importance  is  the  multiplication  formula 
<  theorem  2.1),  which  shows  how  to  express  the  product  of  multiple  integrals  In  a  multiple 
integral  expansion,  thus  providing  a  rudimentary  algebra  for  handling  expansions.  We 
present  the  applications  to  filtering  in  section  3.  In  $3.1,  we  show  that  the  optimal 
filter  can  be  represented  as  the  ratio  of  two  multiple  integral  expansions,  essentially  by 
expanding  the  Kallianpur-Strlebel  formula.  $3.2  addresses  the  issue  of  finding  the  best 
(mean  squai'e)  estimate  of  the  form 


By  combining  the  expansions  of  $3.1  and  the  multiplication  formula,  we  derive  a  system  of 
linear  Integral  equations  for  the  kernels  method  of  analysis  is 
to  transform  measures  to  a  space  on  which  y( •)  is  a  Brownian  process  and  then  to  apply 


the  multiplication  formula  to  discover  the  effect  of  the  Radon-Mlkodym  derivative  so 
Introduced.  The  remaining  sections  apply  these  results,  first  to  rederlvlng  the  Kalman 
filter,  second  to  finding  beat  quadratic  filters. 

It  Is  a  pleasure  to  thank  Professor  3.  K.  Hitter,  for  suggesting  this  problem  and  for 
inspiring  and  guiding  the  research. 

1.2  Filtering  preliminaries 

The  precise  filtering  model  to  be  considered  Is  as  follows.  Let  the  underlying 
probability  space  be  denoted  (n,F,P).  For  0  <  T  <  ■»,  let  {x(t)  I  t  e  [0,T)  }  be 
a  measurable,  real-valued  process  on  (Q,F,P),  h(s,x)  a  Borel  function  on  tO,T]  x  R, 
and  w(t)  a  standard  Brownian  motion  on  (fl,  ,P),  such  that 


Set 


1)  w< •)  Is  Independent  of  x( •) 
11)  B  /*  h^(s,x((>) )ds  <  »  . 


y(t)  -  /g  h(s,x(a))d8  +  w(t)  t  e  (0,T]  . 

Such  a  process  y( •)  will  be  called  an  observation  semi  martingale. 

Let  f(t>  x(8),  s  <  t)  be  a  non-antlclpatlng  functional  of  x( •)  such  that 
ef(t;  x(e),  s  <  t)  <  <»,  V  t  8  [0,T1 ,  and  define  F^  :  -  a{y(s)  |0  <  s  <  t)  and 
:  -  a{x<8),  y<8)  |0  *  8<  t). 

Theorem  1.1  (Kalllanpur,  Strlebel  t9J).  Let 

^2.  .  exp  (-  h(8,x(s))dw(s>  ~  J  .1^  h^(8,x(s))d8) 

Then  (1)  Pff  Is  a  probability  measure,  and  P  and  P^  are  mutually  absolutely 
continuous . 
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(ID  «o{^  I  I''')  -  exPt/o  h(8,x(8))<Jy(B)  -  j  !l  h^(8,x(8))d8]. 


(Ill)  On  ((}>Pg)i  y(<)  l8  a  Brotmlan  motion  Independent  of  x(>). 

(Iv)  x(«)  ha8  the  name  law  on  (((>Pg>  »  (0>P)* 

(v)  E(f(t;x(8),  8  <  t)  1  F^) 

Eg{f  (»x(8)  ,8<t) 

-  - — - 5 -  .  (1.2) 

For  a  nice  treatment  of  this  theorem,  see  Wong  (211.  It  Is  the  principal  theoretical 
tool  for  our  work  In  filtering,  for  it  explicitly  characterises  the  optimal  filter  as  a 
functional  Integral  and  It  establishes  that  y( •)  Is  mutually  absolutely  continuous  with 
Bromlan  motion. 

Finally,  we  remark  that  we  restrict  ourselves  here  to  scalar  processes  only  in  the 
interests  of  notational  simplicity.  The  techniques  to  be  discussed  extend  easily  to  the 
vector  case. 


2.  Multiple  Integrals 
2. 1  Definitions 

The  concept  of  a  multiple  Wiener  Integral  derives  ultimately  from  Wiener's  work  on 
'homogeneous  chaos'  decompositions  of  functionals  of  Brownian  motion;  however,  the  modern 
definition  and  theory  are  due  to  Ito  (7] .  Here  we  will  define  multiple  Integrals  by 
Iteration  of  stochastic  integration.  While  this  differs  from  Ito's  construction,  it  leads, 
as  Ito  (7]  notes,  to  the  same  result  modulo  a  multiplicative  constant.  The  iterative 
definition  is  convenient  for  our  calculations. 

bet  (b(t),F^)  be  a  standard  Wiener  process  with  Its  associated  family  of 
<r*algebras  “  a{b(s)  ;  s  <  t}.  Recall  that,  for  a  jointly  measurable,  F^-adapted 
orocess  ^(t,w)  such  that  E  ((^(s)ds  <  •>,  the  Ito  Integral  (^(s)db(s)  has  the 
properties 


♦ 


E  ^(s)db(s)  -  0  t  <  T 


E(fg  ®  /o  ^ 


L^{tO,T]^)  ”  {f  e  L^(tO,Tl'^)  1  f  is  synmetrlc) 

This  will  be  the  set  of  integrands  for  the  rth  order  integral.  If  f  6  1<^{[0,T)^), 

2  r-1 

f(a>...>  e  L  ((0,Tl  ),  will  denote  the  section  of  f  at  o. 

Definition  2.1  Let  f  e  L^((0,T]')  t  <  T.  I^(f)  is  defined  recursively  by  (L^((0,T]°) 


I^(f)  “  f  for  r  “  0 


”  /«  l*^^f(s..--))<n>{s) 
t  ^  0  s 


I^{f)  la  the  rth  order  multiple  Integral  of  f  with  respect  to  b( •)  up  to  time 
t.  Alternately  stated. 


,s^)db(s^>...db(8.|) 


To  Insure  that  the  rigfit-hand  side  of  (2.3)  is  well  defined  it  suffices  to  show  that 

—  1  2 
'  (f  { 8 , . . . )  )  has  a  jointly  measurable  version  with  bounded  L  (SI  x  (0,T1,  P  x  \)  norm. 
s 

This  may  be  done  by  proving  recursively,  along  with  the  definition,  that 
EI^(f)I^(g)  -  ~  (f,g) 

(2.4) 

”  ^0  =  i . s^)d8^...ds, 

for  all  f,g  6  L^((0,T]^).  This  is,  a  consequence  of  (2.2).  Then,  if  f"  is  a  sequence  of 
symmetrizations  of  separable  functions,  such  that  f"  *  f  in  L^-norra,  (f'''(8, . . . ) )  is 
jointly  measurable  for  all  n  and 


Thus  we  can  find  a  jointly  measurable  version  of  (f(S/.«>))« 

It  la  Important  to  note  that  multiple  integrals  have  sero  man  and  that  integrals  of 
different  orders  are  orthogonal)  that  is,  for  f  8  L^(  [0,Tl ,  g  8  L^{(0,T]**),  q  ^  r,  t. 


El‘(f>  -  0 

(2.5) 

Efl'<f)I^{g))  -  0  . 

These  follow  from  repeated  application  of  (2«1)  and  (2e2)« 

Remarks  The  requirement  of  symmetry  for  the  integrands  is  not  necessary,  since  integration 
is  carried  out  only  over  the  set  where  s^  ^  However  this  convention  is 

convenient  in  formulating  the  multiplication  formula  in  section  2a 2* 

The  following  technical  lemma,  a  Fublni  result  on  Interchanging  db  and  ds 
integrations,  la  needed  latere 


Lemma  2.1  Let  f  8  L^(  (O.T)*^) .  For  t<T 


I^"’(f( . f(u,s,,...s^_,)dudb(8^_^)...db{8^)  .  (2.6) 


Proof  Define 


g*(s,,...8_  ,)  »  /*■  f(u,s  ,...s  ,)du.  The  r.h.s.  of  (2.6)  is  if  N?,.). 

r  1  r*  I  8  -  I  r**  •  t  r 


To  prove  the  lemma,  simply  verify  that 


:(/f  l'^"’(f<s,...))ds  -  lf'’(g*>)^  “  0 
US  r  r 


by  using  the  basic  properties  (2.5)  of  the  multiple  stochastic  Integral. 


For  filtering  applications,  we  must  also  define  multiple  integrals 


Jg  f(8^,...,s^)dy(s^)  •••(dy(s^) 


with  respect  to  observation  semi-martingales 


4 

♦ 


y(t)  »  h{x  )ds  +  w(t) 


(the  assumptions  of  section  1  are  assumed  to  be  in  force).  Such  integrals  are  known  and 
have  been  studied  in  the  context  of  semi-martingale  theory.  However,  the  special  structure 
of  (2.8)  allows  a  simple  definition  which  «*e  present  here.  This  takes  advantage  of  the 
absolute  continuity  of  the  y( •)  process  with  respect  to  Brownian  motion;  as  stated  above, 
if  (0,  F,  P)  is  the  underlying  probability  space,  there  exists  a  probability  measure  Pg 
such  that  Pg  <<  P,  P  «  Pg,  and  y(  •)  is  Brownian  on  (ft,  F,  Pg)*  Therefore,  for 
f  e  ^^([O.T]*^),  we  define  (2.6)  as  the  random  variable,  which  on  (ft,  F,  Pg)  equals  the 
Broimlan  motion  Integral  defined  above,  we  call  this  Integral  ^^1^)  without  reference  to 
measure  or  process,  which  should  always  be  clear  from  context. 

The  iterative  property  of  I^(f)  remains  true  for  dy  integrals;  that  is. 


I^(f)  -  /g  l'”’(f( . . 


where  the  integral  in  (2.9)  la  defined  with  respect  to  the  semi-martingale  y(  •)  in  the 
usual  sense  (see  Llptser  and  Shiryayev  [11]).  However,  neither  the  expression  (2.4)  nor 
the  orthogonality  of  different  orders,  (2.5),  now  holds.  Instead,  we  can  prove  the 
following  lemma,  which  is  useful  in  section  3.2.  (In  this  discussion,  we  abbreviate 
h(s,x(s))  by  h(s)J 

Lemma  2.2  Suppose  h^(s)dsl’^  <  *.  Then  for  k  <  r  and  f  6  L^((0,T]'') 

(i)  E[I^(f)]^  <  ^  independent  of  f 

1* 

(ii)  E  r^(f)  »  /o***/o''  E[h(s^ )  •••h(Sj^))  dSj^  •  • ‘ds^ 


Proof 


We  will  actually  prove  by  induction  the  more  general  result:  for  r  >  1  >  ic 


eCO.Tl 


(2.10) 


where  a  .  e  [0,T1  ^  '') ,  and 
I «  K 


E(h(Oj).-*h<a^^^)I^  (f)) 
K 


“h  r®1  f®k-1 


(2.11) 

/q*'  ®lh<8^)»»h(8^)h(a|^^^)-*h(0j^))d8j^..ds^  . 

Lemma  2.2  is  the  case  f  »  )c  for  every  1c  <  r.  First  we  demonstrate  (2.10)  and  (2.11)  for 
r  >  I  >  )c  «  1,  using  the  iterative  formula  of  (2.9)  and  the  independence  of  x(  •)  and 
w(«).  Thus 


EIh(0j).-h(02)  /Q’f(8)dy(a)l^  =■ 

®l  '’l  5 

E(h(o^)**h<<j2){/g  f(3)h(3)a8  +  /jj  f(3)dw(s)}r  <  (2.12) 

(2E  /J  (h(aj)*-h(3))^ds  +  2  Eth(ajj)^*-h(02)^I  )«E«^  =  ajj^l(a2,**Oj)lfl^  . 


To  derive  the  Inequality  in  (2.12),  the  Cauchy-Schwarz  inequality  is  used  several  times, 
a^  ^  e  l\[0,T)^  ')  for  I  <  r  because  Et/^  h^(s)d8]^  <  Likewise 

<3. 

ECh(aj^)-.h(a2)  /g'f<s)dy^)  - 

o  0.  o 

E[h(  o^)  •  •h(  Cj )  (/g  f(s)h(s)ds  +  /jj  f(3)dw(s)))  “  f  ( 8)B(h(  a^)  • ‘hi  Cj^))  ds  . 


(2.13) 


Now  suppose  (2.10)  and  (2.11)  are  true  for  a  fixed  k  and  all  t,  r  >  t  >  k.  Again, 
using  I^%f)  «  I^(f  (s,  •  •)  )dy(s) ,  Cauchy-Schwar s ,  and  induction 


E(h(0j^)..h(0^^2)I^’  (f)]^  < 

*  E(h(aj^)..h(o^j)h(8,)l^  (f(82,*»))J^d82dB, 


+  2  /jj’'  E(h(Ojj)*»h(aj^^2>*^*<*'**^>l^^» 


*i.k+1<V2'*-'V^' 


By  induction,  a^  e  1,%  (0,T]  Thus  (2.10)  is  true  for  k  +  1.  That  (2.10)  holds 

for  k  also  Inplies 


E  fg  Ig(f{8,-*))ds  < 


Thus ,  from  (2.1), 


E  l''(f (s,  •  •) )dw( 8)  «  0,  tor  t  <  T  . 

'fj  s 


with  the  aid  of  this  equality  we  can  prove  that  (2.11)  also  is  true  for  k  +  1.  This 

completes  the  induction  step.  Induction  stops  at  k  ~  r  since  we  have  required 

,T  2  r 

r  >  t  >  k  in  order  to  apply  EIJ^  h  (s)d8)  <  •>. 
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2.3  The  multiplication  formula. 

AS  above  let  (b(t),F^)  denote  a  standard  Brownian  motion.  If  ^(b(s),  s  <  t)  la  a 
functional  of  b(<)  up  to  time  t.  we  want  to  consider  expansions  of  the  form 


r-0 

(If  4)6  L^(a,F^,P)  such  a  representation  exists,  uniquely,  and  the  series  converges  to 
t|i  In  mean-squarei  see  Ito  (7]  or  Hlda  [5].)  Rules  prescribing  how  this  representation 
changes  as  various  operations  are  performed  on  i|i  must  be  available  if  multiple  integral 
expansions  are  to  be  of  use  In  applications.  In  this  section,  we  address  the  simplest 
problem  in  this  direction.  If  t  e  £^(tO,T]^)  and  g  e  ii^(  (O.Tl'I) ,  what,  if  any,  are  the 
kernels,  that 

l'(f)I^(g)  -  I  (t  <  T)  7 

'  '  1-0  * 


To  express  the  answer,  we  first  introduce  the  following  notation. 
Definition  2. 2 


(1) 


(11) 


Pp  5  projection  of  L^([0,Tl'^)  onto  L^(  [0,TJ '^)  t 
(P^h)(o,...,o^)  ’’‘%(1)'*-'%(r)> 


r 

where  -  permutation  group  on  r  letters. 
For  integers  r,q,k,  0  <  k  <  iiiln(r,q),  and 
f  e  L^((0,T]'^),  g  €  i^(lO,Tl'*) 


(f  V‘'^"‘'l'”'Vq-2k> 


=  ^  /o  '-/o  *'“l'*-'*k'“l'-'Vk>  '»'"l'"“k'  Vk+1'••'’’r+q.2h“*^"*^•1 


(iii)  «  g  =P^^.2^[f  .^(t)gl 
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(Iv)  f  9  9  -  f  9g(t)9  - 


9^(t)  Is  the  operation  by  which  new  kernels  are  created  from  oldi  indeed. 


f  \(t)9  s  X  I,^((0,Tl‘’>  L^([0,Tl’^~^'‘) 


as  the  following  lemma  demonstrates 


Lemma  2.3  For  every  t  <  T 


In  fact. 


f  \(t)g  e  L^dO.Tl*^*’"^’') 


If  ®.(t)gl*  <  c  .  Ifl^  Igl* 


where  q  y  independent  of  f  and  g. 

Proof  It  suffices  to  prove  the  lemma  for  9,  instead  of  9,  since  ^f>q_2k  *  bounded 

operator.  Lot  do  “  **^1  **‘*®r+q-2k'  ”  ‘*“l**‘*“k*  ***  then  have,  using  the  Cauchy-Schwarz 

inequality 

If  .^(t),!  -  /  W2k>’* 


,r+q-Jk  (kl)*  [O.Tl 


<  — i-r  Ifl*  Igl* 
(kl)^ 


To  understand  the  meaning  of  9^(t),  it  is  useful  to  think  of  the  functions  f  and 
g  as  tensors,  which  they  in  fact  are  by  the  isomorphism 

L^dO.Tj*^)  -  L*(tO,T))  L^CIO.T])  (r-fold)  . 

Then  f  9|^(t)g  may  be  viewed  as  a  tensor  contraction  since  it  'sums',  that  is, 
integrates,  f  and  g  along  the  first  k  indices.  Thus  f  9|^(t)g  is  simply  a 
symmetrized,  k-fold,  tensor  contraction.  It  is  in  this  definition  that  the  symmetry  of 
f  and  g  is  usedi  otherwise  tit)  would  have  a  more  complicated  definition.  For 
notatlonal  convenience,  we  shall  often  write  9^^  instead  of  9^{t) ,  in  which  case  the 
(t)  is  to  be  assumed.  Nhen  the  time  parameter  is  important  or  different  than  t,  it  will 
always  be  given. 


I 


W«  can  now  state  the  result 

Theorem  2.1  I.et  f  e  L*(  (0,T] ') ,  g  e  i.^t  I0,Tl'*) .  Then 


rain<r,q) 

i'(f)  i’(g)  -  I  a  (t)q  )  . 

k«0 


(2.14) 


Remerke  1.  (2.14)  shall  Im  referred  to  as  the  multiplication  formula.  Rida  )ias 

independently  derived  this  result  as  an  application  of  his  theory  of  generalised  Brownian 
functionals  (personal  communication  of  T.  Rida;  for  generalised  Brownian  functional  theory, 
see  Rida  (41).  Our  proof  is  elementary,  using  only  Ito's  differentiation  rule.  For 
similar  theory,  see  also  Meyer  (15).  Versions  of  this  formula  are  also  known  in 
mathematical  quantum  field  theory  (Reed,  Simon  [19]).  See  Hitter  and  Oeone  (17]  for 
further  comments. 

2.  The  BRiltipllcatlon  formula  generalises  a  Hermlte  polynomial  indentlty.  nie  nth  order 
Hermlte  polynomial  of  a  single  variable  is 


Imt 


V  ,  V  (-1)"  2„  d"  2^ 

h_(x)  -  —  e  X  /2  -  -  X  /2  . 

n  rl)  S 

dx" 


°r  “  ®  l-*([0,t]')} 


and  let  .  be  an  orthonormal  basis  of  L^((0,t]).  nien,  (Ito  (7],  Itallianpur  (81) 

n  I 


_  ^  I  Pi  Pn  ' 

G,  -  Sp{  n  h  (f:  4.  (s)db(8))  .  ",  ,} 

r  are  pairwise  unequal 


where  Sp  denotes  the  closure  in  L^(P)  of  the  linear  span.  One  then  sees  that  (2.14) 
generalises  the  identity,  ((12]), 


mln(r ,q) 


hr(x)hjx)  -  i;  ' (<^<^ 


k-0 


r+q-2k 


(X) 


(2.15) 
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V  r>q  >  0.  ni«ra  is  a  dlacrspancy  bettMsa  (3.15)  and  (2.14)  in  tha  factors  mltlplyinq  tha 
axpanslon  terms,  but  this  Is  due  to  tha  different  normalisations  involved  in  tha 
definitions  of  h^,  z’^  and  O.  Tha  relationship  bet%«ean  (2.14)  and  (2.15)  may  be  seen 
clearly  in  Hida's  work,  but  we  shall  not  pursue  the  matter  further  here. 

We  will  show  how  to  prove  theorem  2.1  using  ito's  rule  and  induction.  For  this 
purpose,  we  need  certain  facta  and  identities  concerning  0,  and  these  are  collected  in 
the  next  leans.  The  notation  f(S^,..,S^,...)  indicates  t)te  section  of  f  in  which  the 
first  k  variables  are  fixed  at  af,...,S|^  respectively. 

Lemma  2.4 

(i)  f(o^.**)  0^(0, )g(o,,-*)  <02'‘*'Vq-2k-1*  e  L^(  {0,T) 


(ii)  f  0|^(t)g  -  f  O^(o)g  +  /g  f{s,«-)0j^_^(s)g(8,..)ds 
(lii)  For  k  >  1,  <f  ®,{<t)T)<<J,,**,0j^q_2y) 

“  ^7^  0^(t)g  ♦  f  0^(t)g(o,...)J 


“’2'“'Vq-2k> 


(2.16) 


(2.17) 


(iv)  (f  O<t)g)(0,,*«,tf_  )  -  (2.18) 

(■^  0(t)g  ♦  f  0(t)q(o^,**)l  * 

Proof  1)  follo%rs  from  calculations  similar  to  those  in  lenaa  2.3.  The  details  will  not  be 
presented. 


ii)  By  direct  calculation  and  definition,  using  the  symmetry  of  f  and  g  extensively, 
f  0^(t)g 


-  P 

r+q-2k 

•  P 

r+q-2k 

m  P 

'^r+q-2k 


I-  •• 

‘kl  Jq  ^0 


c/'  f"’  — 
‘Jo  Jo 

c—  /®  •• 

‘kl  Jo  Jo 


d8^**dSj^  f(Sj,»»,Sj^,»«)g(s^, 


/q  f(s^,.»,s^,**)g(s^,«*,S|^, 


•)dS|^  ••ds^J 
•  )dS|^«»ds^l 
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I 


^  ’Wq-2kf(k^  ^0  ^ 

-  f  9^lo)q  +  da  «(■,••)  ••) 


(Hi)  and  (Iv).  Hia  proofs  of  (111)  and  (Iv)  ara  aiallar,  (Iv)  baing  just  a  apaelal  casa 
of  (ill).  Wa  ahall  only  praaant  (Iv),  aa  it  la  aiaplar.  Note  flrat  that,  by  definition, 

■~(f(o^,**)  ®  (t)gl  (02,**,o^) 

"  r+q  Tr^FiTT  J-  *‘“l'®»(2)'**“»(r)’  ®‘'’»(r+1)'**'®«(r+q)’  (2.19) 


irtiara  r  e  S_^  ,  la  interpreted  an  a  perautation  of  {2,«>,r-Hi).  Now  using  the  syaaetry 

I 

of  f,  (2.19)  may  be  written  aa: 

,  r 

J,  J-  '‘‘’r(2)'“'‘’i.(j)'  ‘’l'%(j+1)'*‘'‘'i.(r)>  - 


(2.20) 


®it(r+1  )'•*'"»( r+q)‘  • 


nsing  the  expression  analogous  to  (2.20)  for  f  ®  (t)g(o^»»). 


f(o,,**)  ®(t)g  +  f  ®(t)  g(o  ,♦•)}  (o, , »«,  q  ) 


*X  J-  "%(2)'-'%(r^1)‘  ”  ’‘%(r+2)'”'‘»l'”'^(r^)‘’ 

’'^r^q-l 


(7^  J  '‘%(1)'->’'%(r*1)'*-''’r(r+q)‘ 


«  f  9(t)g(  a, ,  ••,0  ._) 

I 
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Thla  la  tha  daairad  raault 


Proof  of  thaocaa  2.t.  Ms  uaa  rto'a  dlffarantlatlon  foraula  and  tha  pracading  laamaa  to 
Implaisant  an  Induction  argusMnt  that  procaada  In  two  atepai 

(a)  Show  (by  Induction)  that  (2*14)  holda  for  ordara  r  «  n,  q  “  1»  Vn. 

(b)  haaumlng  (2.14)  for  (r-1,q),  (r,q-1)  and  (r-1,q-1),  ahow  that  It  holda  for  (r,q). 
(a)  and  (b)  than  provide  a  conalatant  achana  of  Induction  for  proving  theorem  2.1  for  all 
ordara. 

Step  (a)  By  Ito'a  dlffarantlatlon  rule 


f(a)db(a)  /g  g(B)db(a)  -  /g’  (f(a^)g(S2)  +  f(a2)g(a^)]db(s^) 

+  f(a)g(a)da  . 


Thla  provaa  the  caaa  r  •  q  •  1. 

Suppoaa  that  tha  theorem  la  true  for  (r,q)  •  (n-1,1)  and  let  f  e  !.*(  I0,T)  ”) » 
g  e  t^dO.T]).  Applying  rto’a  differentiation  rule  again. 


i^(f)l’(g)  -  g(a)  lj^(f)db(8)  +  /J  l""’  (f(a,«*))l’(g)db(a) 
+  fg  l^”'(g(a)f(a,  ••))da  . 


(2.21) 


By  Induction, 

r"“’(f(a,.*))ll<q)  -  l"(n(f(8,..)  O  g) )  +  l""^(f(8,..)  O  (s)g)  . 

9  S  &  8  1 

LaiWBa  2.5(1)  and  learna  2.1  juatlfy  Interchanging  Integrations  In  tha  last  term  of  (2.21): 


l""’(g(a)f(8,.0)da 


l"”'  { /^  g(u)f(u,8^,  ••,8^^)du) 


Thus,  by  aubatltutlon  In  (2.20) 


I^(«)  l’(9)  -  /$  {I^(g<a)f(-.))  +  l]|(ntf(«,..)  0  gj))db(«) 
+  /p  e^(8)g)db(B) 


-  {g(o^)f(9j.**.o^)  +  ntf(o^,*.)  Og(  )I  (Oj»  ) 

+  i"  0^(o,)g(a2,  ••.o^)  +  /p  g(»)f(8,a^,  ••,o^_^)d8}  . 


By  lenna  2*S  (ill)  and  (iv)  thla  tmcoama 

((n+1)f  O  g)  ♦  l""’  <f  0,(t)g)  , 

which  complat«8  the  Induction  step  of  (a). 

Step  b  Without  loss  of  generality  assume  that  q  <  r.  The  Induction  hypothesis  Is  that 
theorem  2.4  Is  true  for  (r-1/q)/  (r,q-1),  and  (r-1,q-1).  Apply  Ito's  differentiation 


-  /g  l^(g)I^"’(f(8,»0)db(s) 

+  /o  t2”’(9)l3(f)<«>(s) 
t  /j  l^^(f(8,*»))Ig”Vg(8,»»))d8  . 

Next,  use  the  Induction  hypothesis  to  expand  the  Integrands  In  (2.22),  than  Interchange 
ds  and  db(s)  Integrations  where  necessary,  and  collect  like  order  terms.  The  result  Is, 
for  q  <  r 

“  {I**^/*l**“i'**'  o  9l  +  i*^"^)(f  0  g(s^,*»))J(S2,*»,s  )> 


J,  v-,'">H-2--Vq-2k> 


+  fg  *(u,  ej^_^(u)g(u,  ••)du} 


+  I 


^  ••)  e  f(u,»»)  O  ^(u)g(a,  ••)(la} 


To  complete  the  proof,  we  need  only  apply  the  Identities  of  leimna  2.4  (111)  and  (Iv)  to  the 
kernels  of  this  last  expression,  for  example,  the  kernel  of  1  <  k  <  g-1,  equals 

‘'"rV’'*  '75:13k  ''‘-I'"*  V-i>’‘<-2*-> 


+  (/^  f<u,  ••)  Oj^(u)g<u,  ••)du)(S2,  ••)) 


0JJ  '“l  '*1 '  “'•r+q-2k*  *  '^8  ••)*>)  (s^,**) 


This  Is  the  kernel  given  In  (2.14).  The  kernels  of  k  ■  0  and  k  •  q  are 

treated  similarly.  This  completes  the  proof. 


3.  Multiple  Integral  Qcpanslons  in  Filtering  ‘Rteory. 

This  section  explores  the  use  of  multiple  integral  expansions  for  optimal  and 
suboptlmal  filtering.  The  estimation  problem  considered  Is  the  general  problem  stated  in 
the  Introduction,  and  the  notations  and  assumptions  established  there  shall  remain  in 
force.  For  additional  notational  convenience,  we  let  f(t)  :  •  f(tr  x^,  s  <  t),  h(s) 

!  -  h(8,  x(s))  and  f^  i  -  E{f(t)  |  F^). 

3. 1,  Expansion  of  the  optimal  filter 

A 

In  theorem  3. 1  below  we  derive  an  expression  for  f^  as  a  ratio  of  two  multiple 
Integral  expansions  in  which  the  process  of  integration  Is  y(t),  the  observation  semi- 
martingale,  and  the  integrands  are  deterministic  functionals  computable  from  the 
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(uncondltionad)  <U«trlbutlon  of  the  signal  procesa.  First  «m  state  sane  prellmlnery 
definitions  and  a  lewsa. 

Let 

L^  j  -  exp(/g  h(a)dy(s)  -  h*(s)ds]  . 

L^  is  the  important  procesa  in  this  calculation.  Observe  that  and 

0 

(L^,  F*  V  F^)  is  a  suirtingale  on  (fl,  F,  (F*t  •  a{x(s)>  s  e  R*^}).  A  conditioning 

argument  then  shown  that  the  Rallianpur-Striebel  formulai  (1.2),  can  be  expressed  as 


Ro{f(t)Lt  t  F^) 
't  "  - y - 

*0tL,  I  F^} 

The  following  process,  baaed  on  L^,  will  also  appear t 


(3.1) 


,<r) 


-  r/  w. 


i^)....h(Sj^)dy(B^)...4y(s^) 


Note  that  not  a  multiple  integral  of  the  type  defined  in  f2  since  the  Integrand 

is  not  deterministic.  L^*^^  may  be  properly  defined  by  noticing  that 

L^  -  Jg  h(s)L^  dy(o)  . 

Iterative  use  of  the  stochastic  Ito  Integral  then  specifies  L^'^^  for  any  order  r.  This 
is  especially  easy  to  carry  out  on  ((),  F,  i*Q)>  on  which  y(t>  is  a  Brownian  motion 
independent  of  the  signal  (see  theorem  1.1). 

The  following  stochastic  Fubinl  theorem  for  interchanging  conditional  expectation  and 
stochastic  integration  is  needed;  it  is  a  direct  consequence  of  theorem  S.14  in  Llptser  and 
Shlryayev  (11). 

Lemma  3. 1  Let  #(  s)  be  a  f’*^  adapted  process  such  that 
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Bot/J  ♦^(8)d8l  <  -  . 

Then  E(j^  «(>)dy(a)  |  F^>  -  Jg  *<,(♦<«)  I  F|[)dy(e). 

Finally,  It  la  convenient  to  Introduce  the  functions 

t  (t,  s,,...,a  )  t  -  l{f(t)h(B,)...h<8  )}  n  >  0 

n  1  n  in 

k  (a,,. ..,8  )  t  •  E{h(8.)...h(8  >}  n  >  1 

n  I  n  1  n 

ko  :  -  1  . 

Theoreai  3.1 

1)  (Partial  expansion)  If  *(/g  h^(o)do)*^  <  *,  and  E(f^(t)(/p  h^(o>do)*^]  < 

then 


I  i^"Nyt))  +  EjCfCt)  IF^) 

n*0 _ 


I  ryi 


(3.?) 


I  i;"'(k„)  ♦  E„{f(t)  iFp 
n«0 


11)  (Full  expansion)  If  E(exp  h^(s)ds)  <  »,  and  E(f^(t)  exp  h^(8)d8l  <  • 


■t  .2, 


I  iJ^Nyt)) 


4  _  n-0 

't 


(3.3) 


I  I<"»(k„) 
n»0 


and  the  expansions  converge  In  L^(P). 

Proof  I 

Part  1)  By  applying  Ito's  differentiation  rule  to 


dL^  •  h(s)L^du(8) 


so  that 


-  1  ♦  /J  h(8)L^dy(8)  . 


(3.4) 
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Iterating  (4*4) «  %fe  find  that  for  any  r 


“  1  +  /J  h{«)<ly(«)  ♦  /jj’  h(s^)h(B2)dy(82><ly(s^) 


^  •  •  •  •  ♦  I» 


Mow  sttbatitute  this  expansion  Into  the  Kslllenpur-Strlebel  foznula  3.1  for  The 

denominator,  for  example,  becosMs 


n«1 


The  hypothesis  h^(s)ds1*^  <  ••  of  (1)  allows  leimaa  3.1  to  be  applied  to  the  terms  of 

(3.S),  with  the  result. 


^  I  /o***/o"''®o^’**“l*  ***’’^“n*^^^*“n*  *“'*^*•1*  ■*■ 


Since  the  distribution  of  the  signal  process  is  invariant  under  the  change  of  measures 


from  P  to  Pn 


Therefore 


B  (h(8  )»»*h(8  ))  »  E{h(s  )«>«h(s  )} 
u  1  n  in 


«  X  (s  ,...,s  ) 
n  1  n 


A  similar  calculation  yields 


Bg{f(t)L^|F^}  -  5;  (t))  +  B{f(t)L''’|F][) 

n“0 


Substitution  of  these  expressions  into  the  Ksllianpur-Striebel  formula  then  proves  (3.2). 

ii).  Formally,  the  proof  of  the  full  expansion  follows  by  setting  r  «  ••.  To  prove 
it  rigorously,  we  first  show  that  E  h^(s)ds]  <  •  implies 


m.s.(Pg)  lirnd  +  i  /J***/, 
N-M*  n“1 


n-1 


h(s  )«»»h(s  )dy(s  )»**dy(s,)I 
1  n  n  1 


(3.6) 


Denote  the  finite  series  on  the  right  hand  side  of  (3.6)  by  a”.  Then 


N+1 


By  employing  the  standard  computational  rules  (2.1),  (2.2)  for  stochastic  integrals,  this 
last  expression  equals 


s 


V, 


provided  that  it  is  finite.  However, 


Eg(h^(S^)...h^S^)L| 


N+1 


Eg{h^(s^)**«h^(Sj^)  exp(-/jj’’'^’h*(s)ds]  Eulexp[2  /p"*’h(  s)dy(  s)l  IF*  )  }  .  (3.7) 


Now  on  tn,Pg),  x(»)  and  y( •)  are  independent  and  y( »)  is  Brownian,  and  hence,  given 
{x(s),  s  <  s^},  h(a)dy(a)  is  a  Gaussian  random  variable  with  mean  0  and  variance 

f®W+1.2,  _ 

h  (8)d8.  Thus 

S  S 

Egtexp  2  /u”*’h(s)dy(s)|F*  1  -  exp  2  s)ds  .  (3.8) 
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Therefore,  usino  (3.8)  in  (3.7) 


(3.7)  «  Eu{h^(s^)*»«h^(s^^^)  exp(/g***'h^<«)d»)  ) 


eQ{h''(9^)...h'‘(8j,^,)  I  /o*'"'Vo’***/o  • 


As  a  result 


^  ®0^/o***^  h^(8,)...h^{9^)ds^...ds,) 


(3.9) 


Since  E  exp( (s)ds]  <  «,  (3.9)  tends  to  0  as  M  ♦  proving  that  L,.  - 

m.s  (Pq)  lim  A**  for  all  t  <  T,  as  desired.  Lemma  4.1  can  now  be  invoked  for  every 
N+»  ^ 


order  n,  so  that 


Bq{L^|F^}  -  Eg  {ms.  lim  A^IF^} 

-  m.s.  lim  ®o^*t^^t^ 
N-h* 


.8.{P  )lim[1  +  i 
N-M  n“1 


A  similar  proof  expands  Eg {f ( t )L^| F^}  in  the  series 


n«1 


Finally,  to  derive  the  L^ (P)  convergence,  note  that 


0  dPg 


EgL^  “  E(exp  /Jh^(s)d8l  < 
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(ElEotLjF^l  -  (1  +  I  l"(lc^)n 


1  ’• 

Thus,  frcw  (3.6),  E„(L^1F][]  -  (l'(P))  lim  (1  +  T  l"(k  ))  as  claimed.  This  completes 

0  t  t  ^  t  n 


N-M»  n»1 


the  proof  of  theorem  3.1. 


Let  P(A,t|F][)  ”  E[1 .  (x(t) )  |F]^)  denote  the  conditional  distribution  of  x{t)  given 
t  A  t 

the  observation  up  to  time  t. 


Corollary  3. 1  If  K[exp  /Th^(s)ds]  <  • 


p(  A,t|F^) 


E1.(x(t))  +  I  iJIcKI  Jx(t))h(s,)».*h(s  )) 
a  -  t  £1  1  n 


1  +  ^  l"  (Eh(s^>  •..h(s^)) 

n«1 


A  related  formula  is  also  of  interest.  If  x(t)  has  a  density  q(x,t),  x(t)  has  a 
conditional  density  given  by 


E„<L<t)  |Fj^,x(t)-x]q(s,t) 
p(x,t|Fy).-2 - S - - -  . 

^  r*  <  ^  t  I  rj  h 


B|,(L(t)|F^) 


Using  the  same  techniques  as  above,  we  can  easily  derive 


EgCL(t)|FLx(t)-xlq(x,t)  -  (1  +  I  l"(E(h(s,)»**h(s^)|x(t)-x]  )) 

n^O 

K  q(x,t) 


(3.10) 


for  the  numerator  of  p(x,t|F^).  (3.10)  is  often  called  the  unnormalized  conditional 

density. 

Remark!  These  results  all  have  an  obvious  generalisation  to  the  multidlmensinal  case. 

A 

The  Bayes  formula  (4.1)  for  f^  is  properly  viewed  as  the  ratio  of  t%io  conditioned 
functional  integrals,  in  which  the  dependencies  between  x(*)  and  y( •)  are  linked  in  the 
L^  term.  The  expansions  of  theorem  4.1  in  effect  calculate  these  functional  integrals  by 
expanding  I,^.  The  x(*)  and  y(  •)  Interactions  are  then  separated  in  the  sense  that  the 


calculation  of  the  filter  la  decomposed  into  two  partsi  first,  computation,  off-line  and 

prior  to  filtering,  of  the  kernels  t  and  k_,  and,  second,  stochastic  Integration  of 

n  “ 

these  kernels  against  the  observations.  Of  course,  in  actual  practice  one  can  only  compute 
a  finite  nund>er  of  terms.  In  fact,  if  the  kernels  are  aeparable  or  are  approximated  by 
separable  versions,  a  truncated  expansion  may  be  realised  in  a  finite  dlmenslnal  and 
recursive  manner,  because  a  stochastic  differential  system  can  be  constructed  to  realise 
any  multiple  Integral  with  a  separable  kernel.  However,  caution  must  be  excerclsed  in 
approximating  the  optimal  filter  by  truncations  In  (4.3),  because  truncation  of  the  series 
In  the  denominator  can  be  a  source  of  severe  instability.  Although  B{L(t)lF^}  >  0  a.s.,  a 
truncation  approximation  may  pass  through  0  and  ro  lead  to  s  singularity  of  the  filter. 
Thus  an  independent  estimate  of  the  denominator  Is  In  general  required. 

Recently,  attention  has  focused  on  the  unnormallsed  conditional  density  and  the 
corresponding  'unnormallsed*  conditional  moments,  which  are  just  the  numerators  of  the 
Ralllanpur-Strlebel  formula.  R.  Hong  (22]  has  given  a  class  of  Markov  signals  for  which 
analytic  expressions  of  (3.10)  are  available.  Again  truncation  of  (4.10)  will  In  general 
yield  functions  that  attain  negative  values.  For  this  reason,  cumaulant  expansions 


p(x,t|F')  -  e 


have  been  studied  as  an  alternate  source  of  approximate  filters  (see  Rterno  [1]).  He  will 
not  pursue  these  Issues  further,  but  instead  turn  to  other  theoretical  developments  based 
on  therem  3.1. 


3.2  Best  rth  order  filters 

Finite  sums  of  multiple  Integrals  provide  a  natural  class  of  causal  functionals  for 
the  design  of  suboptlMl  filters.  He  Introduce  the  following  definition: 


Definition  3.1 


il)  Tha  beat  rth  order  »atlmaf  of  t( »)  et  timm  t  Is  en  eXeaMnt  ?(t)  6  Y^(t)  auch 
that 

E<f(t)  -  <  B(f{t>  -  b(t))*  (3.11) 

for  all  b(t)  e  Y^(t).  The  kernels  of  f(t),  denoted  by  ag(t),  a^(t),...,a^(t).  are 
called  the  optimal  kernels,  h  process  ?(t)  6  Y^(t),  t  <  T,  satisfying  (3.11)  for  t  <  T 
Is  called  the  best  rth  order  estimate  of  f(«). 

Notice  that  the  best  1st  order  filter  Is  simply  the  linear  filter,  and  thus.  In  the  context 

of  multiple  Integral  expansions,  Isest  quadratic  (2nd  order),  cubic,  quartlc,  etc.  filters 

are  the  natural  extensions  beyond  linear  filtering. 

In  this  section  ««  characterize  the  set  of  optimal  kernels  as  t)ie  solution  to  a  system 

of  linear  Integral  equations.  The  construction  of  these  equations  and  the  proof  of  their 

validity  utilize  the  expansion  formulae  of  theorem  3.1  and  the  multiplication  formula  for 

multiple  Integrals  of  theorem  2.1.  Suppose  for  the  Instant  that  the  full  expansion  (3.3) 

holds  for  the  optimal  filter  and  that  f(t)  -  J  l"(a^(t))  Is  an  element  of  Yj(t),  not 

n»0 

necessarily  the  best.  If  f(t)  Is  to  be  a  good  approximation  of  f(t),  we  want 


f(t)  «  f(t) 


I  I^t.(t)) 


or 

f(t)  I  I^(k  )  -  I  1^(1  (t)) 
J-0  ’  3-0  ’ 


(3.12) 


Now  notice  that  the  left  hand  side  of  (3.12)  can  be  rewritten  as  a  multiple  Integral 
expansion  by  applying  the  multiplication  formula.  In  fact 


f(t)  I  I^k  )  -  I  I^(g.(t)) 
j-0  *  ^  n-0  ^ 


■’  (m,n,i)eCj 


(3.13) 
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where  Cj  >  “  ((m,n,l)  |Bt*'n-2i  •  j,  1  <  *ln(ei,n><  ■  <  r}«  niu«>  one  wey  to  pick  en 
approximation  f  would  be  to  chooae  the  kernels  *„(t>  so  that  9j(t>  matches 
for  as  many  orders  j  as  possible,  la  fact,  this  Is  a  prescription  for  the  optimal 
kernels. 

Theorem  3.2.  Assume  E(/^h^(s)ds)^'^  <  ••  and  X  f t)  (/^*(s)ds)^  <  «  . 


r 

Then  a  best  rth  order  estimate  exists.  It  Is  given  by  f(t)  •  ^ 

n-0  *  " 

gj(t,s^,...,Sj)  -  B{f<t)h(s^>...h(n^>} 

(  •  tyt,s^,....s^)> 


(3.14) 


for  0  <  j  <  r. 

Remark.  The  equations  at  (3.14)  comprise  r  +  1  integral  aquations  for  the  r  *  1 
optimal  kernels  »j(t)  0  <  j  <  r.  This  can  be  seen  from  the  definition  of  gj(t)  and  0 
and  will  be  Illustrated  explicitly  In  the  examples  to  be  discussed. 

Before  proving  theorem  3.2,  tie  first  establish  soam  preliminary  lammas.  The  first 
deals  with  existence  of  estimates. 

Lemma  3.2.  If  ^(/q  h  (8)ds)  <  «•,  then  the  best  rth  order  estimate  exists  and  Is  unique. 

k  2  2 

Proof  From  lemma  2.2  E[I^(8)1  <  2  k  <  r.  Therefore  T^(t)  is  a  mean- 

Is 

square-closed  (Hilbert)  space  of  random  variables.  The  lemma  follows  by  the  projection 
theorem. 

Of  the  next  two  lemmas,  the  first  Introduces  the  optimal  estimate  to  compare 

suboptlmal  estimates,  and  the  second  verifies  a  technical  Identity. 

2  V 

Lemma  3.3.  Let  z,  v  e  L  (!?/', P).  Then 

E(z  -  f(t))^  <  E(v  -  f(t))*  Iff  E(z  -  f(t))^  <  E(v  -  f(t))*  . 

Proof.  Simply  note 


E(z  -  f(t))^  -  E(z  -  f(t))^  +  2E(z  -  f(t))(f(t)  -  f(t)) 
+E(f(t)  -  f(t))* 

-  E(z  -  f(t))^  +  E(f(t)  -  f(t))^  . 
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Proof  ot  th*orw>  3.2  Mea\ia«  of  i< 


3«3  it  •aCfteo*  to  •how  (3«14)  hold*  it  and  only  If 


■  tf(t)  -  f(t))*  <  *tc(t)  - 
for  all  c(t)  e  y^(t)«  Slnca 

■  lc(t)-J(t)l*  -  «(c(t)-f<t)l^  +  B(f<t)-f(t)]*  +  M[c(t)-f(t)](f(t)-f(t)J 
this  will  occur  If  and  only  If 

Kte(t)-f(t)Hf(t)-f(t>]  -0  V  c(t)  c  Y^(t)  . 

nitts,  wa  will  damonatrata  (3.20).  Bagln  by  noting  that 


-1 


(B. 


Yu-’ 


ilLtlF'l) 


Than 


*tc(t)-f(t)l  (f(t)-f(t))  -  E{- 


(c(t)-f(t))If(t)Kg{L^lF^>-E^U(t)t^lFjll 


B„tL^IF^] 


E{*t— |F^)<c(t)-f<t)>(?<t)«p(L^|F^l  -  *p{f(t)l,^|F^n> 


Bg{(c(t)-f(t))(f(t)Eg(L^|F|[l  -  Eptf(t)L^|F^l  ))  . 


Haxt  nota  from  (3.13)  that  gj(t)  dapanda  on  karnola  )c„  ot  at  moat  ordar  ]  't 


f(t)Ep(I,J  iJ(hj)  *  BpLj*'>lF^J 

-  I  I^(g  (t))  +  I  I^(i  (t))  +  f(t)E  {l‘**’>|F^) 

1-0  ^  j-r+1  ^ 


whara  tha  gj(t),  r  -f  1  <  j  <  3r  ara  detormlnad  by  tha  multiplication  formula, 
partial  axpanslons  of  theoraa  3.1,  wa  then  saa  that  tha  axpresalon  f(t)Ep{L^|F^ 
Ep{f(t)i,^|F^}  appearing  In  (3.21)  equals 


(3.20) 


(3.21) 


r.  Thus 


Using  the 


(3.22) 


I  -  ijt))  ♦  I  -  t.(t)) 

j-O  ^  ’  l-rti  ^  ^ 

♦  (9Jt))  ♦  ?(t)» 

J-2r+1  '  ^ 

-  Bg{f(t)L‘*'^F^)  . 

Since  y(*)  is  Brownian  on  (n.P^),  aultiple  integrals  of  different  orders  are  orthogonal 
on  (fl.Pg),  and  so  if  (3.22)  is  used  in  (3.21)  we  find 

(3.21)  -  [Cj(t)-a^j(t)l  (gg(t)-lj,(t))  ■*• 


+  E|j{(c(t)-f(t))f(t)EjtL^^'*|F^n 

-  *^j{(c(t)-f(t))Kp(f(t)t.‘^’^*|F^n  ,  (3.23) 

The  last  two  terms  of  (3.23)  are  sero  by  lemma  3.3.  Thus,  it  is  clear  t)uit  (3.23).  and 
hence  (3.20).  is  zero  iff 

-  tj  0  <  j  <  r  . 

This  completes  the  proof . 

The  technique  of  theorem  4.2  extends  to  other  problems  as  well.  Suppose,  for 
Instance,  that  a  filter 


a*(t)  -  aMt)  ♦  \  l2(a;(t)) 

J-1  ^ 

of  order  q  is  avallsblsi  a'(t)  need  not  be  the  best  qth  order  filter.  Let  r  >  q, 
and.  rather  than  ask  for  the  best  rth  order  filter,  let  us  seek  the  "best  rth  order 
corectlon"  to  a'(t).  l.e..  the  mean'Square  minimising  a(t)  of  the  form 
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r  ^ 

•(t)  -  aMt)  +  £  (t)) 

j-q+1  *  5 

where  aj(t).  j  ■  q  -•■  are  free  to  be  chosen.  Define  the  kernels  gj(t)  as 

before,  but  with  aj(t)  replaced  by  a^(t)  for  0  <  j  <  q. 

Theorem  3.3.  Let  the  hypotheses  of  theorem  4.2  hold.  Then  a(t)  is  the  best  rth  order 
correction  to  a'(t)  if  and  only  if 

q^(t,s^,  •••Sj)  »  B(f(t>h<a^)*«*h(s^)},  q-f1<j<r  .  (3.24) 

Proof.  As  before,  it  suffices  to  show  that  (3.24)  holds  iff 

B(c(t)-a(t)lta(t)-?(t)]  -0 

r  . 

for  all  c(t)  -  a'(t)  +  ^  I^(c.(t)).  By  the  same  calculations  as  in  theorem  4.2 

j-q»1  5 

E(c(t)-a(t)l  (a(t)-f(t)l 


-  Eg{(c(t)-a(t))Ia(t)Ej|{L^|F^)  -  Eg{f(t)L^|F^))) 


■  I  I  ljtq.(t)-t  (t))  ♦  I  lj(9.-t.) 

J-q+1  ’  ^  J-0  '  ’  ^  J-r+l  ’  ^ 

+  I  I^9.(t))+a(t)E„(L‘**'’Fl[l  -  E  {f(t)L‘*'NF]f)J) 

J-2r+1  '  ^  otto  t  t 

r  t  “i-l 

“  I  (c^-a^)(t,s^,*»*,s^)Ig^-i^J(t,s^,-»»,Bj)dSj»».ds^ 


This  equals  zero  iff  gj  •  for  q  4  1  <  j  <  r. 

Remark  Clearly,  an  analogous  result  holds  for  the  case  in  which  an  arbitrary  subset  of 

is  given  and  the  remainder  are  chosen  as  to  optimise  the  mea»>square  filter 

error.  Thus,  if  a^,  j  c  {j, »•••,}  )  c  {0,1, •••,r)  are  given,  then  the  {a.(t)}, 
j  I  q  —  3 

j  /t  )  are  optimally  chosen  iff  g^  •  t.  for  every  J  e  (0, !,•••, r)  - 

1  q  3  3 

As  a  first  example  of  theorem  3.2  let  us  compute  the  kernel  equations  for  the  best 
linear  estimate  f(t)  -  >g(k)  /ga^(t,s)dy( s) .  Prom  (3.13), 
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gp(t)  -  a^Ct)  +  /J«,(t,o)E(h<(i)do 


g,(t,s)  -  •^(t.a)  +  ]^«^<t,a)K[h(«)h(o)ldo  ♦  au(t)Kh(«)  . 
Tha  Xernel  aquations  are  than 

aj,(t)  +  /^a^(t,a)«h<o)da  -  Bf(t) 


a5(t)*h<s)  -  a,(t,a)  +  /pa^(t,o)*(h(a)h(  o)Jdo  -  Bf(t)h(8)  , 

or,  allalnatlng  ag(t)  from  tha  second  aquation, 

ao<t)  +  /Ja^(t,<j)B|h<<T)ldo  -  Bf(t) 

a^(t,a)  +  /^a^(t,o)cov(h(s),h(o)]do  ••  cov[f( t) ,h(s) ] 


(3. 25)  Is,  of  course,  tha  wall-Xnown  Wlanar-Hopf  type  equation  for  optlual  linear 
filtering.  Before  examining  higher  order  examples,  as  will  discuss  the  Kalman  filter. 


3,3  Tha  Kalman  filter 

Consider  the  filtering  problem  in  which  h(t,x)  >  R(t)x  and  x(t)  is  a  Gauss-Markov 
process  arising  as  the  solution  of  the  system 

dx(t)  -  F(t)x(t)dt  *  6<t)db(t) 

where  Xq  •  constant  or  a  Gaussian  m.w,  independent  of  the  Brownian  motion  b( •),  The 
celebrated  Kalman-Bucy  theorem  states  that  the  optimal  state  estiawtor  i^<t)  ■ 

B{x(t)|F^)  satisfies  the  equation 


dx(t)  -  ?(t)x(t)dt  ♦  P(t)H''(t)(dy(t)  -  H(t)x(t)dt] 
x(0)  - 


(3.26) 
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where  P(t)  is  the  solution  of  a  deterministic  Rlccati  equations*  It  follows  that  x(t) 
is,  in  fact,  a  linear  functional  of  y(«);  if  4(t,s)  denotes  the  state  transition  matrix 
of  r(t)  -  P(t)H’'(t)H(t),  the  aolutlon  to  (3.26)  la 


x(t)  -  «(t,0)Xjj  +  *<t,a)P(B)H''^(8)dy(B)  . 


(3.27) 


Thla  almple,  linear  structure  la  not  an  Inanediate  consequence  of  the  expansion  formulae  of 
theorem  3.1,  because,  even  in  this  case,  both  numerator  and  denominator  series  will  be 
truly  infinite  sums.  It  is  therefore  of  interest  to  see  how  x(t)  can  be  derived  from  the 
general  expansion.  We  will  show  that  this  can  be  done  using  theorem  3.2  and  moment 
equalities  for  Gaussian  random  variables. 

The  moat  common  proof  of  the  Kalman-Bucy  filter  Invokes  the  stochastic  differential 
equation  for  the  conditional  moments  (cf.  Fujisaki,  Kalllanpur,  Kunlta  [2]).  In  this 
approach,  the  equation  for  x(t)  requires  knowledge  of  x'^(t),  that  for  x^(t)  knowledge 

A, 

of  X  (t),  and  so  on,  thus  leading  to  an  infinite,  coupled  set  of  equations.  To  derive 
the  Kalman-Bucy  theorem,  it  must  be  independently  argued  that  the  conditional  distribution 
of  x(t)  given  is  Gaussian.  Because  of  identities  between  different  moments  of 

Gaussian  m.v.'s,  this  allows  the  moment  equations  to  be  truncated  at  n  •  2  and  leads  to 
(3.26)  and  (3.27).  By  way  of  contrast,  the  derivation  here  will  not  require  explicitly 
knowing  the  conditional  density.  For  other  methods  of  deriving  the  Kalman-Bucy  filter,  see 
Van  Schuppen  [20]. 

In  the  interest  of  computational  simplicity,  we  will  consider  only  the  most  simple 


dx(t)  -  db(t)  x(0)  -  0 


dy(t)  -  x(t)dt  +  dw(t)  y(0)  -  0  , 


(3.28) 


where  b( *)  and  w( •)  are  independent,  standard  Brownian  motions.  The  techniques  work 
also  for  the  general  case. 


Theorem  3-4  x(t)  •  a(t(e)dy(8)  where  a(t,s>  eetlefiee  the  Wiener-Hopf  eqaation 


a(t,s)  +  a(t,o>  min(s,a)do 


a  t  >  a  . 


Before  preaentln?  the  proof  we  imiat  recall  the  following  anment  identltlea  (Miller  [16], 
Marcua-Hillaky  [14]). 

•  Lemma  3 . 5 ^  Let  [z.,...,Zy}  be  a  jointly  (teusalan  random  vector.  Then 


E[z  ,...,z  ]  -  Bz  Ez  •••z  +  I  cov[z  ,z  ]E[  H  z  ] 

j«2  J  ll'j  * 


Proof  of  theorem  3.4.  Since  y(*)  ia  continuoua  and  Gauaaian,  the  aet  of  polynoatlala  in 
2  V 

y( •)  la  denae  in  L  (Q,F',P),  (Kallianpar  [8]).  Therefore,  it  aufficea  to  ahow  that 
a(t,a)dy(a)  la  the  beat  rth  order  eatimate  for  every  r,  1  <  r  <  ».  Since 


E(^  b^Cajda)"^  < 


E  b^CtX/g  b*(a)d8)'  < 


for  all  r  and  t,  theorem  4.2  applies.  That  la,  if  g^tt,...),  0  <  j  <  •  are  defined 


so  that 


m  m 

a(t,8)dy(s)  I  )  -  I  1^8.) 

i-0  '  ^  i-0  ^  ‘ 

a(t,s)dy(s)  is  the  best  rth  order  estimate  if  and  only  if 

g^(t,8^ ,  )  -  E{b(t)b(8^)»»»bt8j)}  0  <  j  <  r 

From  (3.14),  we  may  easily  calculate 


9o<t)  -  0 


gj(t,»*»)  “  j(a(t,*)  8(t)kj_^)(  •♦••) 

+  (a(t,<)  ®^(t)k^_^^)(  •••)  j  >  0 


(3.29) 


i .  ■ 


•  I*  ' 


-  Vy'  ..v;^  ^:... 


However 


j<a(t,*)  0(t)k^^^)(8^,....,«^) 


"iT  K  '“Vi.)’^ 


I  a(t.«  )*{  n  Ma  >} 
1-1  VI  * 

t<j 


(a(t,’i  0^(t)k^^^)(»^,  •••,a^)  -  a(t,a)C{b(o)b(s^)...b(a^>)do  .  (3.30) 


The  kernel  equations  (3.29)  become 


0  -  Eb(t) 


(3.31) 


a(t,a)  +  a(t,a>C{b(o)b(s) }da  -  K{b(t)b(s)) 


(3.32) 


i  a(t,s.)E{  n  b(s,))  +  a(t,<j)B{b((»)  n  b(s,)}da 


-  B{b(t)b(s^) •••b(s^))  ,  1  >  2 


(3.33)^ 


(3.31)  la  true  by  definition,  and  (3.32),  by  hypothesis.  It  remains  to  prove  that 
(3.33)j,  j  *2  all  hold.  However,  a  direct  application  of  leomia  3.S  shows  that 


B(b(o)  il  b(s  )}  -  \  aln(o,  o.)B{  K  b(s,)) 

• «  *  d  — «  *  «  ^ 


for  every  j.  Using  this,  the  left  hand  side  of  (3.33)j  becomes 


35' 


^  {a(t,a. )  +  /I?  a(t,o)adn<  a,a.  )do}  *{  II  b(t  )} 

1  U  *  * 


■  ^  mln(a  ,t)K{  It  b(B.)) 

1-1  ^  KJ 

W1 


B{b(t)b(a^)«*«b(Sj)} 


whara  tha  flrat  aquallty  anploya  tha  hypothaals  on  a(t>a),  and  the  second  employs  lemma 
3.5  a^aln.  ntus  (3.33).  is  true  for  all  j  >  2. 


3.4  Quadratic  Filters 

As  a  further  example  of  the  technique  of  section  3.2,  «a  will  present  the  optimal 
kernel  equations  for  the  quadratic  case  (r  •  2)  and  sketch  a  theoratleal  approach  to 
their  solution.  To  guarantee  validity  of  the  discussion,  assume  throughout  the  hypotheses 
of  theorem  3.2  for  r  -  2. 

Deriving  the  optimal  kernel  equations  is  simply  a  matter  of  calculation.  Let 
f(t)  -  Sglt)  +  /g  a^(t,s)dy<s)  +  /J  /g'  SjIt.s^.SjIdyCSjIdyls^)  and  let  g^It,***)  be 
defined  from  Bq,  a^,  a2  In  the  manner  Indicated  at  (3.13).  Thus 

gQ(t)  -  a^{t)  +  a,(t)  k,  +  a^It,*)  Oj  kj  (3.34) 


(3.35) 


(3.36) 


g^(t,s)  •  a^(t,B)  +  ag(t)k^(s)  +  (a^(t,»)  0^  kj)(e) 

+  (aj(t,*)  9,  k,)(s)  +  (a2(t,.)  9^  kj)(s) 
g2(t,s^,a2)  -  a2(t,a^,S2)  +  ag(t)kj(s^,S2)  +  (a,(t,0  9  k^)(s^,S2) 
+  (a^(t,<)  9^  kjXs^.Sj)  +  0^  k2)(s^,S2> 

+  Oj  k^)(Sj,S2)  . 


(More  properly,  9  In  (3.34)  -  (3.36)  should  be  0(t).)  According  to  theorem  3.2  f(t) 
Is  optimal  quadratic  Iff  gg,  g^,  and  gj,  are  respectively,  Bf(t),  Xf(t)h(s)  and 
ltf(t)h(Sf )h(S2)>  Moaning  the  definition  of  9(t)  from  section  2  and  kj  - 
Kh(s^) .  •  .h(  Sj) ,  wa  derive  for  the  optlsuil  kernel  equatlonsi 


Ji 


t  * 

Ef{t)  -  a^Ct)  +  /g  (t,a)Eh(8)ds  *  Q  aj(t,«^  ,82)Eh(s^  )h(82)<la2<l9^ 
Ef(t)h(8)  »  a^(t,8)  +  ag(t)Eh(8)  +  a^( t ,  o>Eh(  o)h( 8)do 


(3.37) 


^  ^  o, 

+  Jg  a2(t,8,o)Eh(o)do  +  jg  /g  aj(t,ff^,CJ2)Eh(a^)h(a2)h(e)d8 
Ef(t)h(8,)h(8j)  -  a2(t,8^,82)  +  ag(t)Eh(8,)h(8j)  +  a^ (t,a^ )Eh{82 ) 


(3.38) 


+  a^(t,82)Eh(a^)  +  a^(t,o)E)i(  o)h(8^)h(82)da 

+  /g  [82(1,8^ ,  a)Eh(  o)h(aj )  a^ (t .a^  >  o)Eh(  o)h(a^ )] do  (3.39) 

+  Jg  /g  a2(t,a^,02)E{h(a^)h(8j)h(0,)h(02))d02d0,  . 

Thaae  equations  deserve  some  elenentary  remar)cs  before  we  set  about  solving  them. 
First,  the  optimal  kernels  are  all  Interrelated  In  the  general  case.  We  cannot  solve  for 
ag  and  a^  Independently  of  knowing  »2‘  I<lkewlae,  If  ag  •  Cg,  a^  >  are  the  kernels 

of  the  beat  linear  estimate,  they  will  not,  in  general,  be  the  lower  order  kernels  of  the 
beat  quadratic  estimate.  Secondly,  the  equation  (3.37)  -  (3.39)  can  be  used  for  other 
suboptlmal  designs  In  the  spirit  of  theorem  3.3.  Thus,  if  Sg  and  a  ^  are  given,  and  we 
seek  the  best  quadratic  correction  to  ag(t)  t  a^ (t,s)dy( a) ,  this  will  be  found  by 

solving  (3.39)  for  In  terms  of  a^  and  ag.  The  methods  developed  for  solving  the 

full  set  of  equations  will  also  apply  to  the  best  correction  problem. 

Ag  a  system  of  Integral  equations,  (3.37)  •  (3.39)  looks  complicated  and  contains 
unusual  features.  Nevertheless,  we  will  show  that  solving  the  system  can  be  reduced  to 
two,  familiar  tasks  —  solving  a  linear  estimation  problem  and  solving  a  Fredholm  Integral 
equation.  The  method  behind  this  reduction  Is  simply  to  eliminate  ag  and  a^  to  obtain 
an  equation  for  aj.  The  basic  steps  aret  1)  eliminate  Sg(t)  from  (3.38)  to  derive  the 
Integral  equation  (3.41)  for  a.|i  2)  solve  this  for  a^  in  terms  of  Sj  using  the 
solution  to  the  linear  filtering  problem,  (see  3.42)(  3)  use  (3.42)  to  eliminate  a^  from 
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(3.39)  and  derive  (3.43),  an  Integral  aquation  only  Involving  the  unknown  Uj.  andi  4) 
turn  (3.43)  Into  the  Fredholm  equation  (3.4S).  The  central  equation  Is  thus  (3.45).  Once 
It  Is  solved  for  a^,  a^  and  ag  are  found  by  using  (3.42)  and  (3.37)  respectively. 

Let  R  >  L^([0|tl)  L^((0(tl)  be  the  operator  defined  by 

(R6)(8)  •  cov[h(s),h(o)]  8(o)do  .  (3.40) 


The  first  step  is  easyt  slnply  solve  (3.37)  for  ag(t)  and  substitute  the  result  in 
(3.38).  He  then  derive 


(I  +  R]a,(t,0(s)  -  cov(f(t),h(a))  -  Zh{a)tL^lt.B.a)Ao 

A 

-  /g  Jg  cov(h(8),h(o^)h(o2)la2(t,o^,Oj)dOjdo^  . 

The  next  step,  solving  this  for  a^,  thus  requires  Inverting  I  R. 

Lemma  3.6 

1)  h(a),  8  <  t,  has  a  beat  linear  estimate  h(s)  •  7-  /*  a(s.O/.*v(s) 

(As  a  convention,  set  a(s,a)  •  0  for  0  (  s  <  o  4  t) 

11)  (l+Rl'^-l-Q  where  Q  Is  the  Integral  operator  with  kernel 

q(«1.82)  -  a(8^,e2)  ♦  0(82,8^)  -  a(o,8^)a(o,S2)da 


(3.41) 


0  <  8^,  Sj  4  t  . 

,t  2  4  — 

Proof.  He  are  assuming  E(Jg  h  (8)ds]  <  *.  This  guarantees  that  h(s)  exists,  and,  as 

In  (3.25), 

0(8^, 82)  +  /  a(8^,o)cov(h(S2),h(c))d(j  -  cov[h(B^ )  ,h(  a)l 

04s24s^4t 


11)  Is  standard.  See,  for  Instance,  Geasey  [3]. 

This  lemma  can  now  be  applied  to  solve  (3.41)  for  a^(t,s)i 

a^(t,s)  -  cov(f (t),h(s)1  -  q(8,o)cov(f(t),h(o)]do 


(3.42) 


where 


r' (t.a.o^.Oj)  -  j  cov[h(B),h(o^)h(a2>l 

+  j  (q(a.tf2)^*'(a^)  ♦  q(a,a^)Sh(a'2^} 

+  j  /g  q(a,o)co»(h(o),h(ff^)h(ffj]Jdff  , 

In  derlvlnq  r* ,  advantaqa  waa  taken  of  the  (aaauaMd)  ayaewtry  of  ajCtrB^.aj)  in  a,, 
a2>  flow,  ualnq  (3«37)  and  (3.42),  tre  aiay  elialnate  a^  and  a^  froai  equation  (3.39). 
The  reault  ia 

a2(t.a^,a2)  •  r(t,a^,a2) 

-  /J(f^<a,.o)*2<^'*2'®*  *  r,(a2.o)a2(t,a^,o)]d<i 

-  ^  /o  •^2*’^'  *1 '*2' ®1 '  ®2'*2*^' ®1 '  ®2  ^'*®2**®1 


where 


r(t,a^,a2)  -  covtf(t),h(a^),h(B2)l 

-  /*  cov(h(Bj),h(82)h(o)l  (eovtf(t),h(o)l 


r^(a,a)  -  cov(h(8),h(<j)] 


r2(t,a^,B2.<i,»02^  ”1  (covlh(s^  ),h(B2),h(  0^  )h(  02)1 

-  cov(h(a^),h(B2)lcov(h((j^  ),h(02)n 

-  covih(B^),h(82),h(n)lr'(t,n,o,,02>‘>’'>  • 

It  remains  to  solve  (3.43)  for  82.  This  ia  aimply  a  linear  inteqral  equation  for 
a2.  Ho%«ever,  its  middle  term.  Involving  a  tenaor  contraction  between  a2  and  r^,  is  not 
Btandard,  and  the  usual  linear  integration  theory  does  not  apply  directly.  Despite  this. 
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It  Is  possible  to  rsMTlte  (3<43)  as  a  Fredholm  integral  equation  and  thereby  to  reduce  the 
task  of  calculating  aj  to  a  familiar  problem.  First  notice  that  (3.43)  may  be  rewritten 
in  the  form 


or 


1 


.Sj)  -  (RajCSj, 

-  /o  ^0  ■^2‘“l 


•)>(8^)  -  (Ra2(s^,*))(s2) 

.Sj.o^.Ojja^Co,  ,O2)d02d«^ 


1(1  +  R)a2(a2>*)l  (s^)  -  F(aj,S2)  -  (1(82(8^ ,•)) (s^) 


(3.44) 


In  these  equations,  the  argument  t  has  been  omitted  for  simplicity.  Mow  apply  (I  +  R)~^ 
to  both  sides  of  (3.44).  Again,  an  equation  of  the  form 

((I  -*■  R1a2(s^ ,  •)]  (Sj)  •  linear  terms  in  a^ 

is  obtained,  but  this  time  there  are  no  partial  tensor  contractions  of  the  form  Ra(s^,«) 
(Sj)  on  the  right  hand  side.  With  a  final  application  (I  R)"^  *  I  -  (2  to  both  sides 
the  following  Fredholm  equation  for  Sj  is  derived. 


j(t,S^,S2)  -  r^(t,8^,Sj)  *  Y(t»F^#«2'®1’®2^*2*^'®l'®2^'*®2'’'’l  (3*45) 


where 


|(t,S^,Sj) 


r(t,e^,S2)  - 

+ 


•^0  "2  '  ^  *  1 '  °2  “  1 '  ®2  ^ '  “2  *  ^  *”2'’®) 

q(s,.<»,)q(S2.a2)F(t,a^,C2)<Ja2dc^ 


T^(t,s^,a2,o^ ,02^  •  -  r2(t, 8^, 82,0^,02)  -  q(s2,o^)q(a^iC2) 

+  iS  q<»2»“>f2*^'"i  ,u,  0^,02)'*“  • 
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Remarlta .  The  viewpoint  here  Is  not  recoralve.  Rether  t  le  fixed  throughout  end  Integral 

operators  are  defined  and  Inverted  on  L^( (0,t1 )  or  L^([0,t]^),  and  at  a  later  time  t 

the  whole  operation  would  have  to  be  repeated.  This  poses  an  Interesting  question  for 

further  research.  Hhat  structure  on  the  laoeisnts  Ch(s)«  Kf(t)h(B)>  etc.,  would  allow  a 

recursive  solution  to  the  quadratic  kernel  equations,  in  the  sense  that  a<t  dt,  s^,B2) 

could  be  constructed  in  a  simple  way  from  a(t.a^.S2>7  h  related  question  Is  also 

important,  whan  are  the  solutions  a^  and  a2  aaparable  functions?  If  separability 

1  2 

occured,  then,  as  mentioned  above,  the  stochastic  Integrals  t^(a^)  and  ^2^*2^  could  be 
realized  as  the  outputs  of  stochastic  differential  systeM.  Certainly,  If  F  and  y  of 
the  Fredholm  equation  for  82  are  separable,  a2  will  be  separable,  but  due  to  the 
complicated  manner  In  which  the  moments  Kf(t),  Ef(t)h(s>,  etc.,  combine  to  produce  F 
and  Y<  this  does  not  lead  to  easy  conditions.  This  issue  Is  not  pursued  further. 
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